Fusion-GCN: Multimodal Action Recognition Using Graph Convolutional Networks

In this paper we present Fusion-GCN, an approach for multimodal action recognition using Graph Convolutional Network (GCNs). Action methods based around (GCNs) recently yielded state-of-the-art performance skeleton-based recognition. With propose to integrate various sensor data modalities into a graph that is trained GCN model multi-modal Additional measurements are incorporated the representation either on channel dimension (introducing additional node attributes) or spatial new nodes). Fusion-GCN was evaluated two publicly available datasets, UTD-MHAD- and MMACT demonstrates flexible fusion of RGB sequences, inertial skeleton sequences. Our gets comparable results UTD-MHAD dataset improves baseline large-scale by significant margin up 12.37% (F1-Measure) with estimates accelerometer measurements.